Gradient descent and fast artificial time integration
نویسندگان
چکیده
منابع مشابه
Gradient Descent and Fast Artificial Time Integration
The integration to steady state of many initial value ODEs and PDEs using the forward Euler method can alternatively be considered as gradient descent for an associated minimization problem. Greedy algorithms such as steepest descent for determining the step size are as slow to reach steady state as is forward Euler integration with the best uniform step size. But other, much faster methods usi...
متن کاملStochastic Gradient Descent in Continuous Time
We consider stochastic gradient descent for continuous-time models. Traditional approaches for the statistical estimation of continuous-time models, such as batch optimization, can be impractical for large datasets where observations occur over a long period of time. Stochastic gradient descent provides a computationally efficient method for such statistical learning problems. The stochastic gr...
متن کاملComparative study of Financial Time Series Prediction by Artificial Neural Network with Gradient Descent Learning
Financial forecasting is an example of a signal processing problem w hich is challenging due to Small sizes, high noise, nonstationarity, and non-linearity,but fast forecasting of stock market price is very important for strategic business planning.Present study is aimed to develop a comparative predictive model w ith Feedforward Multilayer Artif icial Neural Netw ork & Recurrent Time Delay Neu...
متن کاملCuriously Fast Convergence of some Stochastic Gradient Descent Algorithms
1 Context Given a finite set of m examples z 1 ,. .. , z m and a strictly convex differen-tiable loss function ℓ(z, θ) defined on a parameter vector θ ∈ R d , we are interested in minimizing the cost function min θ C(θ) = 1 m m i=1 ℓ(z i , θ). One way to perform such a minimization is to use a stochastic gradient algorithm. Starting from some initial value θ[1], iteration t consists in picking ...
متن کاملFast Curvature Matrix-Vector Products for Second-Order Gradient Descent
We propose a generic method for iteratively approximating various second-order gradient steps - Newton, Gauss-Newton, Levenberg-Marquardt, and natural gradient - in linear time per iteration, using special curvature matrix-vector products that can be computed in O(n). Two recent acceleration techniques for on-line learning, matrix momentum and stochastic meta-descent (SMD), implement this appro...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: ESAIM: Mathematical Modelling and Numerical Analysis
سال: 2009
ISSN: 0764-583X,1290-3841
DOI: 10.1051/m2an/2009025